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In a classical measurement the Shannon information is a 
natural measure of our ignorance about properties of a sys- 
tem. There, observation removes that ignorance in revealing 
properties of the system which can be considered to preexist 
prior to and independent of observation. Because of the com- 
pletely different root of a quantum measurement as compared 
to a classical measurement conceptual difficulties arise when 
we try to define the information gain in a quantum measure- 
ment using the notion of Shannon information. The reason is 
that, in contrast to classical measurement, quantum measure- 
ment, with very few exceptions, cannot be claimed to reveal 
a property of the individual quantum system existing before 
the measurement is performed. 

PACS number(s): 03.65.-w, 03.65. Bz, 03.67.-a 



I. INTRODUCTION 

In classical physics information is represented as a bi- 
nary sequence, i.e a sequence of bit values, each of which 
can be either 1 or 0. When we read out information that 
is carried by a classical system we reveal a certain bit 
value that exists even before the reading of information 
is performed. For example, when we read out a bit value 
encoded as a pit on a compact disk, we reveal a property 
of the disk existing before the reading process. 

This means that in a classical measurement the par- 
ticular sequence of bit values obtained can be considered 
to be physically defined by the properties of the classical 
system measured^]. The information read is then mea- 
sured by the Shannon measure of information jjj which 
can operationally be defined as the number of binary 
questions (questions with "yes" or "no" answers only) 
needed to determine the actual sequence of 0's and l's. 

In quantum physics information is represented by a 
sequence of qubits, each of which is defined in a two- 
dimensional Hilbert space. If we read out the informa- 
tion carried by the qubit, we have to project the state of 



* Even in these cases where classical physics instead of def- 
inite measurement results predicts these results with certain 
probabilities, it is still possible at least in principle, to consider 
an ensemble of statistically distributed measurement results 
as revealing corresponding statistically distributed properties 
of the ensemble of classical systems. 



the qubit onto the measurement basis {|0), |1)} which will 
give us a bit value of either or 1. Only in the exceptional 
case of the qubit in an eigenstate of the measurement ap- 
paratus the bit value observed reveals a property already 
carried by the qubit. Yet in general the value obtained 
by the measurement has an element of irreducible ran- 
domness and therefore cannot be assumed to reveal the 
bit value or even a hidden property of the system existing 
before the measurement is performed. 

This implies that in a sequence of measurements on 
qubits in a superposition state a|0)+6|l) (|a|, \b\ ^ {0,1}) 
the particular sequence of bit values and 1 obtained 
cannot, not even in principle^, be considered in any way 
to be defined before the measurements are performed. 
The non-existence of well-defined bit values prior to and 
independent of observation suggests that the Shannon 
measure, as defined by the number of binary questions 
needed to determine the particular observed sequence 
0's and l's, becomes problematic and even untenable 
in defining our uncertainty as given before the measure- 
ments are performed. 

Here we will critically analyze the applicability of the 
axiomatic derivation of the Shannon measure for the case 
of quantum measurement. We will also show that Shan- 
non information is not useful in defining the informa- 



^As theorems like those of Kochen-Specker |2j show, it is 
fundamentally not possible to assign to a quantum system 
(noncontextual) properties corresponding to all possible mea- 
surements. The theorems assert that for a quantum system 
described in a Hilbert space of dimension equal to or larger 
than three, it is possible to find a set of n projection opera- 
tors which represent the yes-no questions about an individual 
system, such that none of the 2 n possible sets of answers is 
compatible with the sum rule of quantum mechanics for or- 
thogonal decomposition of identity Q (i.e. if the sum of a sub- 
set of mutually commuting projection operators is the identity 
one and only one of the corresponding answers ought to be 
"yes"). This means that it is not possible to assign a definite 
unique answer to every single yes-no question represented by 
a projection operator independent of which subset of mutu- 
ally commuting projection operators one might consider it 
with together. If there are no definite (context-independent) 
answers to all possible yes-no questions that can be asked 
about the system then the operational concept of the Shan- 
non measure of information itself, defined as the number of 
yes-no questions needed to determine the particular answers 
the system gives, becomes highly problematic. 
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tion content in a quantum system. In fact we will see 
that when we try to apply Shannon's postulate in quan- 
tum measurements or when we try to define the informa- 
tion content by the Shannon information a certain ele- 
ment emerges that escapes complete and full description 
in quantum mechanics. This element is always associ- 
ated with the objective randomness of individual quan- 
tum events and with quantum complementarity. In the 
end we will briefly discuss a novel and more suitable mea- 
sure of information Yet at first we will return to a 
discussion in more detail of the operational definition of 
Shannon information to quantum measurements. 



II. DISCUSSION OF THE OPERATIONAL 
DEFINITION FOR A SEQUENCE OF 
MEASUREMENTS 

For classical observations Shannon's measure of infor- 
mation can conceptually be motivated through an oper- 
ational approach to the question. We will follow the in- 
troduction of Shannon's measure of information as given 
by Uffink ||] . Consider an urn filled with N colored balls. 
There are n\, ri2, n rn balls with various different col- 
ors: black, white, red. Now the urn is shaken, and 
we draw one after the other all balls from the urn. To 
what extent can we predict the particular color sequence 
drawn? 

Certainly, if all the balls in the urn are of the same 
color, we can completely predict the color sequence. On 
the other hand, if the various colors are present in equal 
proportions and if we have no knowledge about the ar- 
rangement of the balls after shaking the urn, we are max- 
imally uncertain about the color sequence drawn. As no- 
ticed in H one can think of these situations as extreme 
cases on a varying scale of predictability. For example, 
for N=4, there is only one color sequence oooo if all balls 
are white, 4 possible color sequences »ooo, o»oo, oo»o, 
ooo», if there are three black and one white ball in the 
urn, yet 6 possible color sequences ••oo, »o»o, toot, o»»o, 
o»o» and oo»» if there are two black and two white balls 
in the urn. This suggest that the uncertainty we have 
before drawing about the particular color sequence that 
will be drawn is defined by the total number of different 
possible color sequences that are in accordance with the 
given number of balls with their respective colors in the 
urn. 

Consider now a situation where a long sequence of N 
balls are drawn from an infinite " sea" of balls with pro- 
portions pi, P2, p m for the different colours in the 
sea. Then a long sequence contains with high probabil- 
ity about piN balls of the first colour, P2N balls of the 
second colour etc. (such a sequence is called typical se- 
quence). The probability to obtain a particular typical 
sequence (particular colour sequence) is given by M 



p(sequence) = p\ 1 *pf ...p^ N 



2 nh 



where 



h = -y^ Pi log Pi 



(i) 



(2) 



is the Shannon information expressed in bits with the log- 
arithm taken to base 2. Consequently, the total number 
of distinct typical sequences is given by 



W ~ 2 



NH 



(3) 



Suppose now that one wishes to identify a specific color 
sequence of the drawn balls from the complete set of pos- 
sible color sequences by asking questions to which only 
"yes" or "no" can be given as an answer. Of course, the 
number of questions needed will depend on the question- 
ing strategy adopted. In order to make this strategy the 
most optimal, that is, in order that we can expect to 
gain maximal information from each yes-or-no question, 
we evidently have to ask questions whose answers will 
strike out always half of the possibilities. 



2 NH sequences 




FIG. 1. Binary question tree to determine the specific se- 
quence of outcomes (color of the drawn balls) in a sufficiently 
large number N of experimental trials (number of drawings). 
An urn is filled with black and white balls with proportions 
pi and P2, respectively. The expected number of questions 
needed to determine the actual (typical) sequence of outcomes 
is NH, where H = —pi logpi — P2 logp2- 

Since there are W = 2 NH possible different (typical) 
color sequences (all of them have equal probability to be 
drawn), the minimal number of yes-no questions needed 
is just NH. Or equivalently, the Shannon information 
expressed in bits is the minimal number of yes-no ques- 
tions necessary to determine which particular sequence of 
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outcomes occurs, divided by N |g|-|7|. A particular color 
sequence is specified by writing down, in order, the yes's 
and no's encountered in traveling from the root to the 
specific leaf of the tree as schematically depicted in Fig. 
[I] for an explicit example with an urn containing black 
and white balls only. 

If instead of balls with pre-assigned colors we consider 
quantum systems whose individual properties are not de- 
fined before the measurements are performed, does the 
Shannon measure of information still define the informa- 
tion gain in the measurements appropriately? More pre- 
cisely, we ask here the question whether the total number 
W — 2 NH of different possible (typical) sequences of out- 
comes is suitable as a measure of our uncertainty before 
the sequence of quantum measurements is performed. 

In classical physics the behavior of the whole ensem- 
ble follows from the behavior of its intrinsic different in- 
dividual constituents which can be thought of as being 
defined to any precision. This is not the case in quan- 
tum mechanics. The principal indcfiniteness, in the sense 
of fundamental nonexistence of a detailed description of 
and prediction for the individual quantum event result- 
ing in the particular measurement result, implies that 
the particular sequence of outcomes specified by writing 
down, in order, the yes's and no's encountered in a row of 
yes/no questions asked is not defined before the measure- 
ments are performed. No definite outcomes exists before 
measurements are performed and therefore the number of 
different possible sequence of outcomes does not charac- 
terize our uncertainty about the individual system given 
before measurements are performed. 

However, once the sequence of quantum measurements 
is performed and the measurement results are obtained, 
the measure of information needed to specify the particu- 
lar sequence of outcomes realized is defined appropriately 
by the Shannon measure. In the sense that an individual 
quantum event manifests itself only in the measurement 
process and is not precisely defined before measurement 
is performed, we may speak of " generation" of that spe- 
cific information in the measurement. 



III. INAPPLICABILITY OF SHANNON'S 
POSTULATES IN QUANTUM MEASUREMENTS 

As observed by Uffink |^| , an important reason for pre- 
ferring the Shannon measure of information lies in the 
fact that it is uniquely characterized by Shannon's intu- 
itively reasonable postulates. This has been expressed 
strongly by Jaynes || : "One ... important reason for 
preferring the Shannon measure is that it is the only one 
that satisfies ... [Shannon's postulates]. Therefore one 
expects that any deduction made from other information 
measures, if carried far enough, will eventually lead to 
contradiction." A good way to continue our discussion is 



by reviewing how Shannon, using his postulates, arrived 
at his famous expression. He writes Q: 

" Suppose we have a set of possible events whose prob- 
abilities of occurrence are pi,P2, ■■■,Pn- These probabili- 
ties are known but that is all we know concerning which 
event will occur. Can we find a measure of how much 
"choice" is involved in the selection of the event or how 
uncertain we are of the outcome? 

If there is such a measure, say H(pi,p2, ...,p n ), it is 
reasonable to require of it the following properties: 

1 . H should be continuous in the pi . 

2. If all the pi are equal, Pi = ^, then H should be 
a monotonically increasing function of n. With 
equally likely events there is more choice, or un- 
certainty, when there are more possible events. 

3. If a choice be broken down into two successive 
choices, the original H should be the weighted sum 
of the individual values of H. The meaning of this 
is illustrated in Fig. 0. At the left we have three 
possibilities p\ = \, Pi = \, Ps, = \- On the right 
we first choose between two possibilities each with 
probability i, and if the second occurs make an- 
other choice with probabilities |, ^. The final re- 
sults have the same probabilities as before. We 
require, in this special case, that 



h [ 1 -1, 1 -)=h( 1 -, 1 - 

1 2' 3 6 J \2 2 



2 I 3 3 



The coefficient \ is the weighing factor introduced 
because this second choice occurs half the time." 





FIG. 2. Decomposition of a choice from three possibilities. 
Figure taken from [1]. 

Shannon then shows that only the function (||) satis- 
fies all three postulates. It is the third postulate which 
determines the logarithm form of the function and, as we 
will argue, it is this postulate which leads to problems 
when quantum measurements are involved. 

We now turn to the discussion of Shannon's postulates. 
While the first two postulates are natural for every mean- 
ingful measure of information, the last postulate might 
deserve more justification The third Shannon postulate 
originally formulated as an example was reformulated as 
an exact rule by Faddeev MM: For every n > 2 
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H(pi, ..,p n ^ 1 ,q 1 ,q 2 )=H(p 1 , ..,p n - X ,p n )+p n H[ — , — , 

\Pn PnJ 

(4) 

where p n — q x + q 2 . 

Without physical interpretation the recursion postu- 
late (the name was suggested in || ) (0) is merely a math- 
ematical expression which is certainly necessary for the 
uniqueness of the function (||) but has no further phys- 
ical significance. We adopt the following well-known in- 
terpretation |||l0| • Assume the possible outcomes of the 
experiment to be a\, a n and H(j>i, ...,p n ) to represent 
the amount of information that is gained by the perfor- 
mance of the experiment. Now, decompose event a n into 
two distinct events a„A&i and a n Ab 2 ("A" denotes "and", 
thus a A b denotes a joint event). Denote the probabil- 
ities of outcomes a n A b\ and a n A b 2 by q\ and q 2 , re- 
spectively. Then the left-hand side H(pi, ...,p„_i, qi, q%) 
of Eq. (^) represents the amount of information that is 
gained by the performance of the experiment with out- 
comes Oil a„_i, a n A bi, a n A b 2 . 

When the outcome a n occurs, the conditional prob- 
abilities for b\ and b 2 are y- and ^2- respectively and 
the amount of information gained by the performance of 
the conditional experiment is H ^T"Y Hence the re- 
cursion requirement states that the information gained in 
the experiment with outcomes ai , . . . , a„_ i , a n A b\ , a„ A b 2 
equals the sum of the information gained in the experi- 
ment with outcomes 01, a n and the information gained 
in the conditional experiment with outcomes b\ or b 2 , 
given that the outcome a n occurred with probability p n . 

This interpretation implies that the third postulate can 
be rewritten as 

H (p(ai), ...,p(a n -i),p(a n A bi),p(a n A b 2 )) (5) 
= H(p(ai), ...,p(a n -i),p(a n )) + p(a n )H(p(bi\a n ),p{b 2 \a n )) 

where 

p(a n ) = p(a n A b%) + p(a n A b 2 ) 

p(a n A 6i) = p(a n )p(bi\a n ) and (6) 

p(a n A b 2 ) = p(a n )p(b 2 \a n ). 

Here p(bi\a n ) i = 1, 2 denotes the conditional probability 
for outcome a n given the outcome bi occurred and p(a n A 
bi) denotes the joint probability that outcome a n A bi 
occurs. 

If we analyze the generalized situation with n outcomes 
ai of the first experiment A, m outcomes bj of the con- 
ditional experiment B and ran outcomes ai A bj of the 
joint experiment AAB, we may then rewrite the recursion 
postulate in a short form as 

H(AAB) =H(A)+H(B\A) (7) 

where H(B\A) = ^™ p(aj)H(bi\aj, b m \aj) is the aver- 
age information gained by observation B given that the 



conditional outcome Oj occurred weighted by probability 
p(aj) for aj to occur. 

It is essential to note that the recursion postulate is 
inevitably related to the manner in which we gain infor- 
mation in a classical measurement. In fact, in classical 
measurements it is always possible to assign to a system 
simultaneously attributes corresponding to all possible 
measurements, here a^, bj and ai A bj. Also, the interac- 
tion between measuring apparatus and classical system 
can be thought to be made arbitrarily small so that the 
experimental determination of A has no influence on our 
possibility to predict the outcomes of the possible future 
experiment B. In conclusion, the information expected in 
a classical experiment from the joint experiment A AB is 
simply the sum of the information expected from the first 
experiment A and the conditional information of the sec- 
ond experiment B with respect to the first, as expressed 
in Eq. (|). 

Therefore, only for the special case of commuting, 
i.e. simultaneously definite obscrvables, the axiomatic 
derivation of the Shannon measure of information is ap- 
plicable and the use of the Shannon information is justi- 
fied to define the uncertainty given before quantum mea- 
surements are performed. However, in general, if A and 
B are noncommuting observables, the joint probabili- 
ties on the left-hand side of Eq. (||) cannot in princi- 
ple be assigned to a system simultaneously, and conse- 
quently Shannon's crucial third postulate which is nec- 
essary for the uniqueness of Shannon's measure of infor- 
mation ceases to be well-defined. 

Having seen that the third Shannon postulate in gen- 
eral is not applicable in quantum measurements we next 
introduce two requirements that are immediate conse- 
quences of Shannon's postulates and in which all the 
probabilities that appear are well-defined in quantum 
mechanics. We will show that the two requirements 
are violated by the information gained in quantum mea- 
surements implying that the Shannon measure loses its 
preferential status with respect to alternative expressions 
when applied to define information gain in quantum mea- 
surements. 

1. Every new observation reduces our ignorance and 
increases our knowledge. In his work Shannon Q| 
offers a list of properties to substantiate that H is a 
reasonable measure of information. He writes: "It 
is easily shown that 

H(AAB) < H{A)+H(B) 

with equality only if the events are independent 
(i.e., p(ai A bj) = p(ai)p(bj)). The uncertainty of a 
joint event is less than or equal to the sum of the 
individual uncertainties" . He continues further in 
the text: "... we have 

H(A) + H(B) > H(A AB) = H(A) + H(B\A). 
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Hence, 



H(B) > H{B\A). 



(8) 



The uncertainty of B is never increased by knowl- 
edge of A. It will be decreased unless A and B are 
independent events, in which case it is not changed" 
(we have changed Shannon's notation to coincide 
with that of our work). 

2. Information is indifferent on the order of acquisi- 
tion. The total amount of information gained in 
successive measurements is independent of the or- 
der in which it is acquired, so that the amount of 
information gained by the observation of A followed 
by the observation of B is equivalent to the amount 
of information gained from the observation of B fol- 
lowed by the observation of A 



H(A) + H(B\A) = H(B) + H(A\B). 



(9) 



This is an immediate consequence of the recursive 
postulate which can be obtained when we write the 
recursion postulate in two different ways depending 
on whether the observation of A is followed by the 
observation of B or vice versa. An explicit example 
for a sequence of classical measurements is given in 
Fig. |. 

Are these two requirements satisfied by information 
gained in quantum measurements? Consider a beam of 
randomly polarized photons. Filters Ft, F450 and ,F«_> 
are oriented vertically, at +45°, and horizontally respec- 
tively, and can be placed so as to intersect the beam of 
photons (Fig. ^). If we insert filter i*j the number of 
photons observed at the detection plate will be approx- 
imately half of the number in the incoming beam. The 
outgoing photons now all have vertical polarization. No- 
tice that the function of filter Fj cannot be explained as a 
" sieve" that only lets those photons pass that are already 
of vertical polarization in the incoming beam. If that 
were the case, only a certain small number of the ran- 
domly polarized incoming photons would have vertical 
polarization, so we would expect a much larger attenua- 
tion of the beam of photons as they pass the filter. 

Denote with A and B properties of the photon to have 
polarization at +45° and horizontal polarization, respec- 
tively. If FU is inserted behind the filter Fj we are cer- 
tain that none of the photons will pass through (Fig. |I|a). 
For a photon with vertical polarization we have complete 
knowledge of the property B, i.e. H(B) = 0. Notice 
that a "sieve" model could explain this behaviour. If we 
now insert F450 between Fj and F w we observe an effect 
which cannot be explained by a sieve model where the fil- 
ter does not change the object. However we now observe 
a certain number of photons at the detection plate (about 
j of the number of photons in the beam passed through 



Ft) as shown in Fig. |]b. In this case our knowledge of 
the property B is not complete anymore. 

The acquisition of information about property A there- 
fore leads to a decrease of our knowledge about prop- 
erty B, i.e. H(B\A) > 0. Note that on the photons 
absorbed by the filter F450 we cannot measure prop- 
erty B subsequently. However already for the subenscm- 
ble of the photons passing through the filter our uncer- 
tainty about property B becomes larger than implying 
= H(B) < H(B\A) which clearly violates requirement 
(@). Another example of sequence of quantum measure- 
ments where requirement (^) is violated is given in Fig. 
||. Clearly, violation of the requirements (||) and (||) oc- 
curs when the corresponding operators A and B do not 
commute. 
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FIG. 3. Indifference of information to the order of its ac- 
quisition in classical measurements. A box is filled with balls 
of different compositions (plastic and wooden balls) and dif- 
ferent colors (black and white balls). Now, the box is shaken. 
In (a) we first draw a ball asking about the color of the drawn 
ball and gain //(color) = 1 bit of information. Subsequently, 
we put the black and white balls in separate boxes, draw a ball 
from each box separately and ask about the composition of the 
drawn ball. We gain fln(comp.) = bits for the black balls 
and H w h{comp.) — 1 bit for the white balls. In (b) we pose 
the two questions in the opposite order. We firstly ask about 
the composition of the drawn ball and gain //(comp.) = 0.81 
bits. In a conditional drawing we ask about the color of the 
drawn ball and gain H mo (color) = bits for wooden balls and 
H p i (color) = 0.92 bits for plastic balls. The total information 
gained is independent of the particular order the two questions 
are posed, i.e. //(color) + 1/2Hm (comp.) + l/2// m (,(comp.) 
= //(comp.) + l/4Z/ mo (color) + 3/4// pi (color) =1.5. 
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What is the origin of the violation of the require- 
ments (^) and (|^) in quantum measurements? In con- 
trast to a classical measurement which just adds some 
new knowledge to our knowledge at hand from the previ- 
ous measurements, in a quantum measurement the gain 
of the new knowledge is always at the expense of ir- 
recoverable loss of complementary classes of knowledge. 
This originates from the distinction between "total" and 
"complete" information in quantum physics. In classi- 
cal physics the total information about a system is com- 
plete. In quantum physics the total information of a sys- 
tem, represented by the state vector, is never complete 
in the sense that all possible future measurement results 
are precisely defined^]. In fact, the total information of a 
quantum system suffices to specify the eigenstate of one 
nondegenerate (with one-dimensional eigenspaces only) 
observable only. 

For example, the state of a photon passing through fil- 
ter Ft is specified by the complete knowledge about the 
property A of vertical polarization. If we let a photon in 
this state pass through filter F450 as given in Fig. ||d, our 
knowledge of the photon changes, and therefore its rep- 
resentation, the quantum state, also changes. The total 
information of a photon in the new state is completely 
exhausted in specifying property B of polarization at 45° 
and no further information is left to also specify property 
A, thus implying unavoidable loss of the previous knowl- 
edge about this property. This further implies that the 
set of future probabilistic predictions specified by the new 
projected state is indifferent to the knowledge collected 
from the previous measurements in the whole history of 
the system. Such a view was assumed by Pauli who 
writes^]: "Bei Unbestimmtheit einer Eigenschaft eines 



*Yet, we do not hesitate to emphasize that it certainly is 
complete in the sense that it is not possible to have more infor- 
mation about a system than what can be specified in its quan- 
tum state. In fact, the state vector represents that part of our 
knowledge about the history of a system which is necessary to 
arrive at the maximum possible set of probabilistic predictions 
for all possible future observations of the system. For exam- 
ple, a set of complex amplitudes of a ^-function is a specific 
representation of the catalog of our knowledge of the system. 
This view was assumed by Schrodinger Jllj who wrote: "Sie 
((die i/)-Funktion )) ist jetzt das Instrument zur Vorausage 
der Wahrscheinlichkeit von MaBzahlen. In ihr ist die jeweils 
erreichte Summe theoretisch begriindeter Zukunfterwartun- 
gen verkorpert, gleichsam wie in einem Katalog niedergelegt. 
Translated: "It (the ?/>-function) is now the means for predict- 
ing the probability of measurement results. In it is embodied 
the momentarily attained sum of theoretically based future 
expectation, somewhat as laid down in a catalog. " 

§In translation: "In the case of indefiniteness of a property of 
a system for a certain experimental arrangement (for a certain 
state of the system) any attempt to measure that property 



Systems bei einer bestimmten Anordnung (bei einem bes- 
timmtcn Zustand des Systems) vernichtet jeder Versuch, 
die betreffende Eigenschaft zu messen, (mindestend teil- 
weise) den EinfluB der fruhcren Kcnntnisse vom System 
auf die (eventuell statistischen) Aussagen fiber spatere 
mogliche Messungsergebnisse." This clearly makes pos- 
sible to violate requirements (|8|) and (^) in quantum mea- 
surements. 
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FIG. 4. The gain of knowledge in a new observation reduces 
our knowledge at hand from a previous observation. Filters 
Ft, F450 and F+-, are oriented vertically, at +45° and hori- 
zontally, respectively. If filter F<-> is inserted behind the filter 
Fj , no photons are observed at the detector plate (Fig. a) . In 
this case our knowledge about horizontal polarization (prop- 
erty B) of a photon passing through filter Fj is complete. If 
filter F450 is inserted between F^ and F«, a certain number 
of photons (1/4 of the number of photon passing through Fj) 
will be observed at the detection plate (Fig. b). Now acquisi- 
tion of information about the polarization at +45° (property 
A) leads to the decrease of our previous knowledge about hor- 
izontal polarization of the photon. 

Here a certain misconception might be put forward 
that arises from a certain practical point of view. Accord- 
ing to that view, for example, complementarity between 
interference pattern and information about the path of 
the particle in the double-slit experiment is considered 
to arise from the fact that any attempt to observe the 
particle path would be associated with an uncontrollable 
disturbance of the particle. Such a disturbance in itself 
would then be the reason for the loss of the interference 
pattern. In such of view it would be possible to define 
Shannon's information for all attributes of the system si- 



destroys (at least partially) the influence of earlier knowledge 
of the system on (possibly statistical) statements about later 
possible measurement results." 
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multaneously, and the third Shannon postulate, as well 
as the requirements (0) and (||), would be violated be- 
cause of the unavoidable disturbance of the system oc- 
curring whenever the subsequently measured property B 
is incompatible with the previous one A. Yet, this is a 
misconception for two reasons. 
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FIG. 5. Dependence of information on the order of its ac- 
quisition in successive quantum measurements. A spin- 1/2 
particle is in the state |z+) spin-up along the z-axis. Spin 
along the x-axis and spin along the direction in the x-z 
plane tilted at an angle a from the z-axes are succes- 
sively measured, in one order in (a) and in the oppo- 
site order in (b). Whereas we obtain an equal portion 
//(cos 2 (-7r/4 — a/2), sin 2 (-7r/4 — a/2)) of information in the 
conditional (subsequent) measurement both in (a) and in 
(b), the amounts of information //(cos 2 a/2, sin 2 a/2) and 
f/(i, |) = 1 we gain in the first measurement in (a) and in 
the first measurement in (b) respectively, can be significantly 
different. Specifically for a-tOwe have complete knowledge 
about spin along the direction at the angle a in (a) but ab- 
solutely no knowledge about the spin along the x-axis in (b). 
We emphasize that we do not assume any specific functional 
dependence for the measure of information H. 

Firstly, as theorems like those of Bell (ll| or Green- 
berger, Horne and Zeilinger juj show, it is not possible, 
not even in principle, to assign to a quantum system si- 
multaneously observation-independent properties which 
in order to be in agreement with special relativity have to 
be local. We therefore cannot speak of a " disturbance" in 
the measurement process if there are no objective prop- 
erties to disturb. 

Secondly, over the last few years experiments were con- 
sidered and some already performed, where the reason 
why no interference pattern arises is not due to any un- 
controllable disturbance of the quantum system or the 



clumsiness of the apparatus. Rather the lack of interfer- 
ence is due to the fact that the quantum state is prepared 
in such a way as to permit path information to be ob- 
tained, in principle, independent of whether the experi- 
menter cares to read it out or not. One line of such re- 
search considers the use of micromasers in atomic beam 
experiments ]l5| , another one concerns experiments on 
correlated photon states emerging from nonlinear crys- 
tals through the process of parametric-down conversion 

The view that complementarity must be based on 
the much more fundamentally property of mutual ex- 
clusiveness of different classes of information of a quan- 
tum system was assumed by Pauli [ [l2| in the analysis 
of the uncertainty relations^: " ... diese Relationcn 
enthalten die Aussage, dafi jede genaue Kenntnis des 
Teilchenortes zugleich eine prinzipiclle Unbestimmtheit, 
nicht nur Unbekannthcit des Impulses zur Folge hat und 
umgekehrt. Die Unterschcidung zwischen (prinzipicllcr) 
Unbestimmtheit und Unbekanntheit und der Zusammen- 
hang beider Begriffe sind fur die ganze Quantentheorie 
entscheidend." 



IV. DIFFICULTIES IN DEFINING THE 
INFORMATION CONTENT OF A QUANTUM 
SYSTEM 

To define the information content of a physical sys- 
tem one might consider different measures of informa- 
tion. However only those measures of information have 
physical significance according to which the defined in- 
formation content of the system possesses properties 
which naturally follow from the physical situation con- 
sidered. These properties are, for example, invariance 
under changes of the modes of observation of the sys- 
tem and conservation in time if there is no information 
exchange with an environment. We show now that the 
information content of a quantum system, if it is assumed 
to be measured by the Shannon measure of information, 
cannot be defined in any way to have these properties. 

The classical world appears to be composed of particles 
and fields, and the properties of each one of these con- 
stituents could be specified quite independently of the 
particular phenomenon discussed or of the experimental 
procedure a physicist chooses to determine these proper- 



din translation:"... these relations contain the statement 
that any precise knowledge of the position of a particle implies 
a fundamental indefiniteness, not just an unknownness, of the 
momentum for a consequence and vice versa. The distinction 
between (fundamental) indefiniteness and unknownness, and 
the relation between these two notions is decisive for the whole 
quantum theory." 
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ties. In other words the properties of constituents of the 
classical world are noncontextual. 

In particular, the total lack of information about a 
classical pointlike system (with no rotational and inter- 
nal degrees of freedom) defined as Shannon's informa- 
tion associated with the probability distribution over the 
phase space is independent of the specific set of vari- 
ables chosen to describe the system completely (such as 
position and momentum, or bijective functions of them) 
and conserved in time if there is no information exchange 
with an environment (i.e. if the system is dynamically 
independent from the environment and not exposed to 
a measurement)]^. Operationally the total information 
content of a classical system can be obtained in the joint 
measurement of position and momentum, or in successive 
measurements in which the observation of position is fol- 
lowed by the observation of momentum or vice versaf^. 

Contrary to the classical concepts most quantum- 
mechanical concepts are limited to the description of phe- 
nomena within some well-defined experimental context, 
that is, always restricted to a specific experimental pro- 
cedure the physicist chooses. In particular the amount 
of information gained in an individual quantum measure- 
ment depends strongly on the specific experimental con- 
text. In the optimal experiment when the measurement 
basis \i) coincides with the eigenbasis of the density ma- 
trix p of the system: p\i) — Wi\i) the amount of infor- 
mation gained is maximal (See for example ||). Since 
in the basis corresponding to the optimal experiment the 
density operator is represented by a diagonal matrix with 
elements Wi, the information gain defined by the Shan- 
non measure equals the von Neumann entropy as given 
by0 



H 



Wi log 



w. 



-Tr(plogp). 



(12) 



This has the important property to be invariant under 
unitary transformations p — > UpU + . The invariance un- 
der unitary transformations implies invariance under the 
change of the representation (basis) of p and conservation 
in time if there is no information exchange with an en- 
vironment. The later precisely means that if we perform 
the optimal experiments both at time to and at some fu- 
ture time t, Shannon's information measures associated 
to the optimal experiments at the two times will be the 
same, i.e. 

H(t) = - ^2 Wi(t) log Wi(t) = -^WilogWi = H(t ). 

i i 

(13) 

Here, the eigenvalues of the density matrix at time t are 

Wi(t) = Wi. 

However, without the additional knowledge of the 
eigenbasis of the density matrix p we cannot find the 
optimal experiment and obtain directly the Shannon in- 
formation associated. Also, all the statistical predictions 
that can be made for the optimal measurement are the 
same as if we had an ordinary (classical) mixture, with 
fractions iw, of the systems giving with certainty results 
that are associated to the eigenvectors \i). In this sense 
the optimal measurement is a classical type measurement 
and therefore in this particular case, and only then, Shan- 
non's measure defines the information gain in a measure- 
ment appropriately***. Considering also our previous 



discussion it is therefore not surprising that Shannon's 



^ Given the probability distribution p(r,p, t) over the phase 
space the total lack of information of a classical system is 
defined as Q 

H total {t) = - / d 3 fd 3 pp(f,P, t) log (10) 
J P(r,p) 

where a background measure p(f,p) is an additional ingredi- 
ent that has to be added to the formalism to ensure invariance 
under change of variables when we consider continuous prob- 
ability distributions. The conservation of H to tai in time for 
a system with no information exchange with an environment 
is implied by the Hamiltonian evolution of a point in phase 
space. 

H In full analogy with (^) we may write H to tai(r,P) = H(f) + 
H(p\r) = H(p) + H(r\p). 
5§ For a given density matrix p the von Neumann entropy 

S(p) = -Tr(plogp) (11) 

is widely accepted as a suitable definition for the informa- 
tion content of a quantum system. For a system described in 



TV- dimensional Hilbert space this ranges from log TV for a com- 
pletely mixed state to for a pure state. The von Neumann 
entropy has the important property to be invariant under uni- 
tary transformations. However, we observe that any function 
of the form Tr(f(p)) (the operator f(p) is identified by having 
the same eigenstates as p and the eigenvalues f(u>j), equal to 
the function values taken at the eigenvalues Wj of p.) pos- 
sesses this invariance property. We also observe that the von 
Neumann entropy is a property of the quantum state as a 
whole without explicit reference to information contained in 
individual measurements. 

*** Consider a situation where instead using of single systems 
to send information to the receiver a sender uses a sequence 
of TV systems where each individual system is drawn from an 
ensemble of pure states { | -0i ) , \ip n )}, with frequency of oc- 
currence {wi, ...,w n } respectively. It was shown in |lq| that 
for sufficiently large TV there are 2 NS ^' highly distinguish- 
able sequences of pure states which become mutually orthog- 
onal as TV — > oo. Here S(p) = — Tr(p log p) is the von Neu- 
mann entropy and p = y\ Wi\%pi) (ipi\. This means that if the 
sender uses a sequence consisting of a choice of states that 
respects the a priori frequencies Wi, and the receiver distin- 



ct 



measure is useful only when applied to measurements 
which can be understood as classical measurements. 

Which set of individual measurements should we per- 
form and how to combine individual measures of infor- 
mation obtained in the set in order to arrive at the infor- 
mation content of a quantum system if we do not know 
the eigenbasis of the density matrix? Quantum comple- 
mentarity implies that the total information content of 
the system might be partially encoded in different mutu- 
ally exclusive (complementary) observables. These have 
the property that complete knowledge of the eigenvalue 
of any one of the observables excludes any knowledge 
about the eigenvalues of all other observables. Such a set 
of observables for a spin-1/2 particle can for example be 
spin components along orthogonal directions. 

We consider now a quantum system described in n di- 
mensional Hilbert space and we denote a complete set of 
m mutually complementary observables 'ft by {A, B, ...}. 
The property of mutual expansiveness implies that if the 
system is in an eigenstate of one of the observables, for 
example, in the eigenstate \aj) of the observable A and 
we measure any other observable from the set, say B, 
projecting the system onto states {|&i), \bi), \b n )}, 
the individual outcome is completely random (all mea- 
surement results are equally probable) 



(aAbi 



1 
n 



Vm'. 



(14) 



It was shown in fll9|| that the density matrix of the 
system can fully be reconstructed if one performs a com- 
plete set of mutually complementary observations. This 
suggest that the total information content of a quantum 
system represented by a density matrix p is all obtainable 



guishes whole sequences rather than individual states, then 
the (Shannon) information transmitted per system can be 
made arbitrary close to S(p). Here again the total density 
matrix p N of N systems can be made arbitrary close to the 
one as if we had a classical mixture of the 2 JVS ^ sequences 
of states. 

T ^ T To specify a system described by a n x n density ma- 
trix completely one needs n 2 — 1 independent real numbers. 
Any individual, complete measurement (we consider here only 
complete measurements, i.e., where operators associated to 
the measurements are without degeneracy) with n possible 
outcomes defines n — 1 independent probability values (the 
sum of all probabilities for all possible outcomes in an in- 
dividual experiment is one). Therefore, just on the basis of 
counting the number of independent variables, we expect that 
the number of different measurements we need in order to 



determine the density matrix completely is 



n + 1. 



Ivanovic [0 , and Wootters and Fields [Q demonstrated the 
existence of exactly n + 1 mutually complementary observ- 
ables by an explicit construction in the cases of n prime and 
n = 2 k . 



from a complete set of mutually complementary mea- 
surements. To obtain the total information one however 
cannot perform the set of measurements successively be- 
cause, unlike the classical case, the information obtained 
in successive quantum measurements depends on the or- 
der of its acquisition (see Fig. || and discussion above). 
Instead it seems that any attempt to obtain the total in- 
formation content of a quantum system has to be related 
to the complete set of mutually complementary exper- 
iments performed on systems that are all in the same 
quantum state. 

We suggest that it is therefore natural to require that 
the total information content in a system in the case of 
quantum systems is sum of the individual amounts of 
information over a complete set of m mutually comple- 
mentary observables. As already mentioned above, for a 
spin-1/2 particle these are three spin projections along 
orthogonal directions. If we define the information gain 
in an individual measurement by the Shannon measure 
the total information encoded in the three spin compo- 
nents is given by 

H to tai :=H 1 (p+,p-)+H 2 (p+, p-)+H 3 (p+,p-). (15) 

Here, e.g. p£ is the probabilities to find the particle with 
spin up along direction x. 

Considering now an explicit example we will show that 
the total information H to tai based on the Shannon mea- 
sure is in general not invariant under unitary transfor- 
mations. We calculate ( |l5| ) for a spin-1/2 particle in the 
state \ip) = cos9/2\z+) + sin6/2\z— } and we find that 



H 



total 



(16) 



1 - sin 9 , 1 - sin 1 + sin 6 . 1 + sin ( 

o l0 § n n l0 S o 



- cos - log ^cos - J - sin - log I sin - j + 1 

depends on the parameter 9, thus being not invariant un- 
der unitary transformations. This associates a number of 
highly counter- intuitive properties to H to tai- 1) it can be 
different for states of the same purity (e.g. it takes its 
maximal value of 2 bits of information for 9 = and 
it takes its minimal value of 1.36 bits for 9 = 7r/4); 2) 
it changes in time even for a system completely isolated 
from the environment where no information can be ex- 
changed with environment; 3) it can take different values 
for different sets of the three orthogonal spin projections. 
These unnatural properties we see again as a strong indi- 
cations for inadequacy of the Shannon measure to define 
the information gain in an individual quantum measure- 
ment. 
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V. A SUGGESTED ALTERNATIVE MEASURE 
OF INFORMATION 

We suggest that it is natural to require that the in- 
formation content of the quantum system defined as a 
sum of individual measures over a complete set of mu- 
tually complementary measurements is invariant under 
unitary transformations. Having shown that this cannot 
be achieved with the Shannon measure of information we 
now introduce a new measure of information that differs 
both mathematically and conceptually from Shannon's 
measure of information and according to which the in- 
formation content has invariance property. 

The new me asur e of information is a quadratic function 
of probabilities 



i(pi 



(18) 



and it takes into account that for quantum systems the 
only features known before an experiment is performed 
are the probabilities for various events to occur (See 
for discussion; there a specific normalization factor in 
expression ( |l8| ) was used resulting in maximally k bits 
for n = 2 k possible outcomes). The measure I(pi, ■■■,p n ) 
takes its maximal value of (n — l)/n if one pi = 1 and it 
takes its minimal value of when all pi are equal. 

The important property of the new measure of infor- 
mation is that the total information defined with respect 
to it is invariant under unitary transformations. Using 
Eq. ( |l4|) one obtains that the sum over individual mea- 
sures of information of mutually complementary obser- 
vations results in vM 



Itotal ■= X/ (Pi' 

i=i 



(19) 



EE 

j=i i=i 



Trp 2 - -, 
n 



the j-th observable. The total information content of the 
system therefore might all be encoded in one single ob- 
servable or, alternatively it might be partially encoded in 
all m mutually complementary obscrvables. For a com- 
posite system in a product state the total information 
can all be encoded in individual systems constituting the 
composite system or, alternatively in the extreme case 
of maximally entangled states it can all be encoded in 
joint properties of the systems with no information left 
in individual systems B . 

Independent of the various possibilities to encode infor- 
mation the total information content of the system can- 
not fundamentally exceed the maximal possible amount 
of information that can be encoded in an individual ob- 
servable [— (n — 1)/^]. This upper limit is reached when 
the system is in the pure state. When the system is in 
a completely mixed state the total information takes its 
minimal value of 0. 

The property of invariance under unitary transforma- 
tions implies that the total information content of the 
system does not dependent of the particular set of mutu- 
ally complementary observables; it is a characterization 
of the state of the system alone, not of the specific ref- 
erence set of complementary observables. Furthermore, 
since evolution in time is described by a unitary operation 
the total information of the system is conserved in time if 
there is no information exchange with the environment. 

We would like to note that the total information ( |l9| ) 
was used in |2f| to study the transfer of entanglement and 
information for quantum teleportation of an unknown en- 
tangled state through noisy quantum channels. The total 
information ( |l9| ) belongs to the set of quantum counter- 
parts of nonextensive entropies finding its application in 
increasing number of problems in quantum physics, e.g. 
description and controlling of laser cooling [|26|, a non- 
extensive approach to the decoherence problem [^7j , de- 
scription and quantifying of entanglement, and deducing 
criterions for separability of density matrices |2qj29]. 



for a system described by the density matrix p. Here p\ 
denotes the probability to observe the z-th outcome of 



Expressions of the general type of Eq. ([l8|) were studied in 
detail by Hardy, Littlewood and Polya pi|. They introduced 
a general class of mathematical expressions 



for < a < oo 



(17) 



that from the point of view of information theory all can 
be assumed to quantify information properly. These expres- 
sions are also closely related to Tsallis's p3] nonextensive 
entropy S a = j4^I]" =1 (pf - 1) and Ranyi's (23) entropy 



CONCLUSIONS 

In this work we have stressed some conceptual diffi- 
culties arising when Shannon's notion of information is 
applied to define information gain in a quantum measure- 
ment. In particular we find that the axiomatic derivation 
of Shannon's measure of information is not applicable in 
quantum measurements in general. We also show that 
the information content of a quantum system defined ac- 
cording to Shannon's measure possesses some strongly 
non-physical properties. We argue that these difficulties 
in defining the information gain in quantum measure- 
ment by the Shannon measure of information arise when- 
ever it is not possible, not even in principle, to assume 
that attributes observed are assigned to the quantum sys- 
tem before the observation is performed. 
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Having critized Shannon's measure of information as 
being not appropriate for identifying the information gain 
in quantum measurement we proposed a new measure of 
information in quantum mechanics that both mathemat- 
ically and conceptually differs from Shannon's measure 
of information. While Shannon's information is appli- 
cable when measurement reveals a preexisting property, 
the new measure of information takes into account that 
for quantum systems the only features known before an 
experiment is performed are the probabilities for various 
events to occur. In general, which specific event occurs 
is objectively random. 

The total information content of a quantum system 
defined according to the new measure of information as 
the sum of the individual measures of information for 
mutually complementary observations is invariant under 
unitary transformations. This implies that the total in- 
formation content of the system is invariant under trans- 
formation from one complete set of complementary vari- 
ables to another and is conserved in time if there is no 
information exchange with an environment. 
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